Introduction
Since the beginning of the digital age, more efficient and safer storage of digital information has been strived for. Computer memory has standardised upon data structures being stored in memory and defined using fixed size. Encryption has concentrated upon mathematical complexity in block based ciphers.
The Field of Data Management
Creating, structuring, controlling and managing data across all disciplines is not recognised as a single area of professionalism today. Professionals tend to be categorized by the specific solution area or discipline in which they operate and not truly in the field in which they operate. Areas of skill are shrinking in their scope. The more skills required for a person to be considered a skilled or professional person in a field, the more they need to learn before they are productive. As skills and knowledge grow the requirement for ever more specialization is based upon the quantity of skills combined with the average person's ability to learn it. This invention challenges this narrowing, in that the invention covers a technology that brings many fields together back into a single field of data structure and management.
Memory Structure
Storage together with the processes of checking and manipulating digital information in the memory of a computer is a grouped binary based logic; grouping into blocks of 7, 8, 16, 32 or 64 bits etc., in the most common systems today. This method of storing and visualizing digital information not only affects the efficiency of some processing operations but also has forced the current standard approach for software developers and system analysts, and as a consequence establishes the typical way by which programming development tools work to resolve many of the technical computing requirements of today.
The memory in a computer is currently based upon a binary data definition and is 2 dimensional in structure and definition. A “1” located in the bit position represents a value based upon the binary column position. In binary a “1” in the first column is 2 to the power of 0; in the second it is 2 to the power of 1; in the third it is 2 to the power of 2 and so on. Therefore within a byte of 8 bits a total value range between 0 and 255 can be represented. This is the most efficient usage of the space possible.
Current Compromise
Electronic computers, as with other embedded systems, establish and define the method of structure, form and subsequent operational use and efficiency of digital information. The structure is implemented similarly in all computers and has served us well. However, as with many technical implementations compromises are often needed to create a solution. Computers and computing devices have this fixed approach and by consequence, have compromises in some areas where information is used, stored or processed. The future, and the substance of this invention, is to deliver an alternate method of storing and manipulating information that brings performance benefits and functional benefits in the situations where the current methods are a compromise.
New Demands on Encryption
Furthermore, the need for encryption and keeping secrets, while being established way back in the past, only really exploded commercially relatively recently. Prior to commercial microchip manufacture that enabled the world of computing to exist for average people and average businesses: wars, conflict and the military had always driven the need for safe encryption systems. Ironically, however, the very availability of this computing and the subsequent success in the “Performance Race” actually made each existing and valuable solution in the history of recent cryptography, shine only for a limited time until computer processing performance caught up and forced systems to go into retirement. The quantity of digital information that people and companies store and manipulate has grown exponentially over the last decade. The internet, music (mp3), digital video encoding and DVD's have all done their bit to contribute to this growth. The Internet not only enables communication, but also provides access to the largest library of information ever available to man and that within only a few years of it coming into existence. Music, Video and other forms of media have been seen as a cost saving opportunity for publishers and distributors but have equally put these businesses at risk. As this increase in demand has grown and made the production of memory and disks cheap, so have the new requirements grown. Ideas that were unthinkable a few years ago are not only possible but are being done. The mapping of DNA and other large storage hungry applications have become possible, at least with regard to storage. Yet the structure for storage and method for manipulating digital information has been treated inherently, as set in stone. Few have created new storage philosophies to aid processing methods, save perhaps the area of compression for space saving.
Historical Encryption
The development of encryption for commercial purposes has meant that various systems have been established as de facto standards.
DES (Data Encryption Standard) was developed in the mid 1970s; used mainly for commercial application; and broken by cryptanalyst's within 25 years. The message was encrypted block by block, so the process is described as a block cipher. Originally the encryption/decryption key was 56 bits long, but this was increased after a successful attack by cryptanalysts. An enhanced system described as Triple DES was used with longer keys. The American National Institute for Science and Technology invited the submission of new algorithms to replace the vulnerable DES. In 2001 the AES (Advanced Encryption System) was introduced. It was a block cipher process and used much longer keys than DES. However computer power is increasing rapidly, and much money is being invested in the development of quantum computers. If they are successful they will dramatically increase the power available to cryptanalysts for exhaustive key searches.
Where asymmetric (public key) encryption is used, keys are mostly much larger, and cryptanalysts attempt to attack the underlying mathematical formula. RSA for example is an important public key encryption method. This is also vulnerable to attack and there are attempts to increase the complexity of the algorithm and lengthen the key.
Stream ciphers encrypt the message letter by letter or bit by bit. One of the strengths of stream ciphers is that they lack error propagation. An erroneous encryption of one bit does not affect subsequent bits. The weakness is that if the plain text is discovered and the corresponding cipher text is known then the key stream sequences may be deduced. It is difficult to manage the distribution of the key stream and indeed to generate it in the first place from shorter seed keys. Stream ciphers are appropriate for data sent as continuous stream and used by the recipient as a stream, e.g. films and music. Block ciphers on the other hand offer a higher level of security and are typically used in financial transactions. In such usage organising data into blocks does not present a problem.
Statistical Testing
Statistical testing forms a fundamental component of the assessment of block ciphers. If an attacker is conducting an exhaustive key search, then there should be no indication that they are near to the correct key. Several methods have been introduced to strengthen block ciphers. There are two standard ways known as cipher feedback mode and cipher block chaining mode. This is another way in which cryptographers make their algorithm more complex in order to protect against more powerful cryptanalysis.
Current Encryption
The one-time pad is a stream cipher with a key stream that is random. This is generally taken to be the only way to achieve perfect secrecy. However there is a practical problem associated in distributing the key and therefore it is deemed to be impracticable. Commercial encryption is relatively new and is most often implemented using block ciphers with mathematically induced disruption. The leading authors and encryption organisations define the following functional requirements for modern cryptographic systems: integrity, the protection against messages being changed; non-repudiation, the ability to prove that the sender sent the message; authentication, to prove that the sender is who he purports to be.
Processing Data Efficiently
Storing, using and manipulating digital information today does not in itself have a problem. It does work. However as larger memory and storage hungry systems place ever-greater demands upon processing performance and energy requirements the enthusiasm to move into new areas is more often dampened because the physical hardware technology is not yet sufficiently mature or capable. In some cases the foundation thinking behind computing is that a job is done or not done, and the answers to questions are exact. This does not actually reflect much of the real world. While the tax office would not take kindly to an invoice approximating the amount due for sales tax, industries that use approximations and statistics can indeed accept a compromise, clearly within tolerances, that are both useful and where needed lawful. A current computer being applied to the problem of determining a difference between say two large strands of DNA would with traditional thinking need to process each segment or block and compare it with its relative equivalent in the other being compared against. Of course upon the detection of a difference the looping process can stop and the processing energy, time and performance need not be wasted continuing the check, however a change can occur anywhere and therefore the entire check could theoretically be performed. The principles of organising and storing digital information to allow a processor to be more efficient in performing certain operations is per se probably not new but the system this patent covers certainly is.
Current Inconsistencies in Thinking
Since cryptography has been of interest commercially, there has been a very clear tendency to follow one specific path of thinking with a pair of overlapping but closely allied philosophies. Cryptographic techniques can be broken into two basic camps: “Stream” based systems and “Block” based systems. “Stream” based systems are fast in the opinion of leading edge development in this area, but fail to provide as safe an encryption basis as the modern world requires today. “Block” ciphers are slower but are deemed to be safer. Cryptography also makes a fundamental distinction between the manner in which a cryptographic system goes about “hiding” or “disturbing” the message, namely: true random based disturbance or mathematical induced complexity based disturbance. Currently the only commonly accepted absolute safe system in existence is the One-Time-Pad (OTP) or Vernam Cipher under the condition that a key is only used once and is random. This OTP system is a “Stream” based system using true random as its source for disturbance. All current modern systems being employed today are “Block” based encryption with mathematically induced complexity based disturbance. This fundamental difference of what has been proven to work being one hundred and eighty degrees out of synchronisation with the drive of current professional thinking raises more questions than it answers. However the importance of this change in direction is directly relevant to the system in this patent and to the characteristics that are simply not achieved with current approaches and implementations of encryption systems. For the purpose of completeness it is worth pointing out that the OTP system is unrealistic for most implementations because of practical limitations and not limitations in its ability to provide absolute security.
Random Data and Use
Mathematicians and engineers use random data for statistics and testing. A data stream is defined as random if it is unpredictable. While this definition is commonly accepted, it is not easy to derive a test to confirm that data is random. One way is to see if the data can be compressed, since true random would offer no patterns or repetition to allow it to be represented in a shorter form. Random sources of information are needed for statistics and other related mathematical tasks. Mathematicians define random as being information that is unpredictable. While it is a definition that is perfect it does not imply an easy or achievable method of turning out a resulting tool to test. For this purpose the world of compression is an excellent method of testing randomization. Compression aims to identify patterns in data and use patterns to represent the data in a shorter form. Ultimately random information can be deemed so if a person, tool or other mechanism cannot determine a pattern and neither can they predict the information that comes next.
Encryption Conclusion
In summary, cryptology is more important now than ever it was. The Internet offers an opportunity for those determined to discover information needed to be kept secret. Solutions to date are mere palliatives. Encryption techniques based on current thinking lean towards greater complexity and key size. One weakness which has yet to be addressed in the design of current encryption systems is that no single part of a system should contain unnecessary information. Such unnecessary information would give someone the ability to derive knowledge that they should not have. The solution to this is to either follow the trend of increasing complexity, or to devise a system which does not depend on complex mathematics. Such a system should ideally make it impossible to succeed with an exhaustive key search whilst additionally rendering it impossible to yield the message from the cipher text without the key, thus making the system totally secure. Furthermore, commercial encryption is relatively new and is most often implemented in the form of a “Block Cipher” with a mathematical induced disruption. The leading authors and encryption organizations set in stone the following additional functional requirements for modern cryptographic systems: Integrity, the protection against messages being changed; Non-repudiation, the ability to prove the sender sent the message; and Authentication, to prove that someone else has not pretended to be the sender. We believe these additional functional requirements are not only unnecessary but also dangerous. We base this on the knowledge that a One-Time-Pad has none and is absolutely safe.
The present application also includes consideration of a characteristic that is not mentioned in the leading publications of today namely: Authorization, the ability to derive knowledge of only such parts of the system as are necessary for a system to work. This additional functional requirement is fundamental to such systems as Digital Rights Management (DRM), because the aim of DRM is to attempt to provide a decrypt key to the purchaser at the same time as not allowing the purchaser to know the key. This currently appears impossible and in the absence of Authorization, it is impossible. Therefore, DRM currently concentrates upon complicated and manufacturers' trust based systems in addition to embedding lock keys in playback machines. However, the future must be to create encryption that does not rely upon mathematical description nor is it able to be mapped, and thereby disabling all computers, current and future, as a tool for “cracking” encryption.